feat(study): MEG data-sample validation study (M100, memory-bounded, resumable)#12
Merged
Merged
Conversation
…resumable) Adds a meg_validation study type — the first heavy / real-data study built on the stateful study pipeline. Validates the auditory M100 evoked-response claim (magnetic peak ~80-120 ms) against a bounded sample of an open MEG dataset, as five resumable checkpointed steps: fetch_sample -> preprocess -> epoch -> evoked -> validate_finding. - engine/meg_study.py: the five steps + a memory-bounded real mne backend (_MneIO) and an injected-I/O contract (MegIO). Heavy deps (mne/numpy) are imported ONLY inside backend methods — the module and the whole plugin import fine with neither installed. The real backend never full-preloads: preload= False, single run, gradiometer subset, crop, downsample; each step reloads the prior step's on-disk artifact, so an OOM/restart resumes from the last completed step instead of re-fetching. validate_finding emits supported (peak in window) / refuted (clearly outside) / inconclusive (degenerate sample), with measured latency + amplitude as evidence. - tools.py: wire meg_validation into matilde_study_create (default plan + params: dataset_id, expected_window_ms, bounds) and matilde_study_run dispatch, mirroring the bibliography dispatch; handlers stay thin. - tests: stdlib + fakes (no mne/numpy/network/data) — step state transitions, verdict boundaries (100ms supported / 300ms refuted / empty inconclusive), custom window, and resumability proven with call counters (fail at evoked, resume, earlier steps not re-run). Plus an opt-in live test (MATILDE_LIVE=1 + mne) on bst_auditory, skipped otherwise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
juniperbevensee
added a commit
to NimbleCoAI/hermes-swarm-map
that referenced
this pull request
Jun 22, 2026
…ation study) (#109) v0.4.0 adds the memory-bounded resumable MEG data-sample validation study (NimbleCoOrg/Matilde#12). Reapply to deployed Matilde to expose it. Co-authored-by: Juniper Bevensee <juniperbevensee@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it does
Adds a
meg_validationstudy type — the first heavy / real-data study built on the stateful study pipeline. It validates the auditory M100 evoked-response claim (a magnetic field peak ~80–120 ms after an auditory stimulus) against a bounded sample of an open MEG dataset (bst_auditory), as five resumable, checkpointed steps:fetch_sample → preprocess → epoch → evoked → validate_findingvalidate_findingemits a finding: supported if the measured peak latency is within the expected window (default 80–120 ms), refuted if clearly outside, inconclusive if the sample is degenerate (no measurable peak). Evidence = measured latency + amplitude.Lazy-mne + injected-I/O testing approach
mne/numpyare imported only inside the real backend's methods (_MneIO), never at module top.matilde_plugin.engine.meg_study— and the whole plugin — imports cleanly with neither installed (verified in a venv with no mne/numpy).grepconfirms no top-level heavy imports anywhere inengine/.MegIOhandle. Production passes none and gets the real backend; tests pass fakes — no mne, no numpy, no network, no data files, fully deterministic and stdlib-only._TEST_IOhook so the offline dispatch test stays stdlib-only; production leaves it unset (real lazy-mne).Memory-bounded by design
The real backend never full-preloads a recording:
preload=False, one run, a gradiometer channel subset, crop to a stimulus-locked window, downsample — then persists small intermediates. Each step reloads the prior step's on-disk artifact rather than holding live objects, so a memory kill or container rebuild resumes from the last completed step instead of re-fetching the whole chain.Resumability evidence
A unit test forces a failure at the
evokedstep: the study goesblockedwithfetch_sample/preprocess/epochdoneand no finding emitted. Onresumewith a working backend, call counters prove the earlier steps are not re-run (fetch/preprocess/epoch = 0 calls), only the failed step and the remaining one execute, and the study completes with the expected finding.Tests
tests/test_meg_study.py(step transitions, verdict boundaries 100ms→supported / 300ms→refuted / empty→inconclusive, custom window, resumability) andtests/test_meg_study_tools.py(create+run dispatch through the tools).MATILDE_LIVE=1+ mne importable, viapytest.importorskip): runs the real bounded pipeline onbst_auditoryand asserts a peak in a plausible window. Skipped without the flag/mne — that's expected.Do not merge — leaving open for independent audit.